{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 神经网络的实践"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 小批量梯度下降\n",
    "\n",
    "\n",
    "## TensorDataset \n",
    "\n",
    "TensorDataset在PyTorch中是一个非常有用的数据集类,它可以将tensors打包成数据集以供模型训练。\n",
    "\n",
    "可以看到TensorDataset将输入的特征和标签打包成了数据集,可以用于后面的模型训练。\n",
    "\n",
    "它的主要作用有:\n",
    "\n",
    "- 将特征和标签组合成数据集\n",
    "\n",
    "- 支持打包多个tensor,如图像、标签等\n",
    "\n",
    "- 可以索引样例,方便读取\n",
    "\n",
    "- 兼容 DataLoader,可以批量读取\n",
    "\n",
    "TensorDataset使得tensor数据的组织和读取非常简洁高效。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## DataLoader\n",
    "\n",
    "在PyTorch中,DataLoader是用于数据读取的重要类,其主要作用和用法示例如下:\n",
    "\n",
    "1. 数据读取\n",
    "\n",
    "DataLoader实现了对Dataset按batch读取,支持多进程读,自动转为GPU等功能:\n",
    "\n",
    "\n",
    "2. 批处理\n",
    "\n",
    "可以指定batch大小,将数据分成批进行读取:\n",
    "\n",
    "\n",
    "3. 随机打乱\n",
    "\n",
    "设置shuffle=True可以按epoch随机打乱数据:\n",
    "\n",
    "4. 多进程加速\n",
    "\n",
    "设置num_workers启动多进程读取数据:\n",
    "\n",
    "5. 样本采样\n",
    "\n",
    "可以通过Sampler自定义从数据集中采样样本:\n",
    "\n",
    "总之,DataLoader提高了数据读取效率,是PyTorch中使用数据集的标准方式。\n",
    "\n",
    "* for batch_idx, (x,y) in enumerate(batchdata):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<img src=\"20.png\"/>"
      ],
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.display import Image\n",
    "Image(url= \"20.png\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<img src=\"22.png\" width=\"600\"/>"
      ],
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Image(url= \"22.png\", width=600)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "from torch import nn\n",
    "from torch import optim\n",
    "from torch.nn import functional as F\n",
    "from torch.utils.data import TensorDataset\n",
    "from torch.utils.data import DataLoader\n",
    "\n",
    "import torchvision\n",
    "import torchvision.transforms as transforms\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "features = torch.randn(100, 5) \n",
    "labels = torch.randint(0, 10, (100,))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([5, 1, 9, 9, 3, 5, 7, 9, 2, 5, 2, 8, 4, 5, 4, 3, 1, 8, 5, 6, 9, 3, 1, 7,\n",
       "        4, 2, 4, 4, 4, 1, 7, 9, 4, 0, 6, 9, 2, 5, 1, 2, 8, 0, 6, 9, 4, 7, 4, 3,\n",
       "        3, 4, 6, 7, 8, 7, 1, 8, 6, 1, 1, 6, 1, 5, 6, 8, 7, 3, 1, 7, 7, 9, 0, 1,\n",
       "        9, 4, 6, 2, 1, 1, 7, 2, 9, 7, 5, 0, 3, 8, 4, 7, 0, 1, 3, 2, 9, 8, 0, 3,\n",
       "        5, 5, 5, 7])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset=TensorDataset(features, labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[tensor([[-9.5280e-01, -3.0774e+00, -5.4249e-01, -3.0317e-03, -1.4348e-01],\n",
      "        [ 9.2463e-01, -6.3370e-01, -9.1456e-01,  4.8292e-01,  1.3892e-01],\n",
      "        [-4.6048e-01,  1.3321e+00, -2.5969e-01, -3.4334e-01, -5.5015e-01],\n",
      "        [ 8.0044e-01,  5.3746e-01, -8.6799e-03, -4.4420e-01,  1.2717e+00],\n",
      "        [-6.0578e-01, -1.3198e+00,  1.2482e+00, -5.5186e-01, -8.4056e-01],\n",
      "        [ 2.5463e+00, -9.4506e-01,  6.3811e-02,  6.4624e-02, -7.4463e-02],\n",
      "        [-8.9326e-01, -2.1968e-01, -2.5476e-01, -7.5230e-01,  2.1866e-01],\n",
      "        [-1.9377e+00, -9.8148e-01, -4.5201e-01, -9.4173e-01, -5.1840e-01]]), tensor([9, 3, 5, 2, 5, 2, 6, 8])]\n"
     ]
    }
   ],
   "source": [
    "for i in DataLoader(dataset, batch_size=8):\n",
    "    print(i)\n",
    "    break"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 初始化特征和标签张量\n",
    "features = torch.randn(100, 5) \n",
    "labels = torch.randint(0, 10, (100,))\n",
    "\n",
    "# 用TensorDataset打包特征和标签\n",
    "dataset = TensorDataset(features, labels)\n",
    "\n",
    "# 创建数据加载器\n",
    "dataloader = DataLoader(dataset, batch_size=8) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "x=torch.tensor([[1.0,2.0,1.5],[3.0,4.0,2.3],[5.0,6.0,1.2]])\n",
    "z=torch.tensor([0,2,1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.manual_seed(40)\n",
    "x = torch.rand((500,20),dtype=torch.float32)\n",
    "y = torch.randint(low=0,high=3,size=(500,1),dtype=torch.float32)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([0., 1., 2.])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "#y.view(x.shape[0]).long()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 回归与二分类的实现"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Model0(nn.Module):\n",
    "    def __init__(self,in_features=10,out_features=2):\n",
    "        super(Model0,self).__init__()\n",
    "        self.h1=nn.Linear(in_features,100,bias=True)\n",
    "        self.h2=nn.Linear(100,30,bias=True)\n",
    "        self.out=nn.Linear(30,out_features,bias=True)\n",
    "\n",
    "    def forward(self, x):\n",
    "        h1_out=self.h1(x)\n",
    "        h1_out_r=torch.relu(h1_out)\n",
    "        h2_out=self.h2(h1_out_r)\n",
    "        h2_out_r=torch.relu(h2_out)\n",
    "        out_out=self.out(h2_out_r)\n",
    "        out_out=torch.softmax(out_out,dim=1)\n",
    "        return out_out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.3898, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.3722, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2352, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.3073, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2657, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2596, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2560, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2432, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2592, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2650, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2533, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.3136, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2509, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2469, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2322, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2918, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2462, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2732, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2681, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.3003, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2553, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2492, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2449, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2487, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2477, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2905, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2448, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2376, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2569, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2497, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2529, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2546, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2432, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2546, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2632, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2574, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2812, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2485, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2471, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2416, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2823, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2422, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2698, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2692, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2825, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2517, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2486, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2442, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2462, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2466, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2818, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2423, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2375, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2578, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2492, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2502, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2538, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2431, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2524, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2638, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2576, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2706, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2473, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2476, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2463, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2774, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2402, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2669, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2694, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2727, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2501, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2489, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2438, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2451, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2455, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2769, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2409, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2375, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2582, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2489, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2486, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2536, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2433, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2511, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2640, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2573, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2644, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2467, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2479, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2486, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2742, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2394, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2653, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2684, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2676, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2490, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2497, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2428, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2443, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2450, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2735, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2400, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2373, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2583, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2488, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2469, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2540, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2431, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2504, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2643, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2577, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2599, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2463, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2479, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2501, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2717, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2379, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2641, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2690, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2616, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2492, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2505, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2418, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2453, grad_fn=<MseLossBackward0>)\n",
      "tensor(0.2458, grad_fn=<MseLossBackward0>)\n"
     ]
    }
   ],
   "source": [
    "lr=0.1\n",
    "gamma=0.5\n",
    "epochs=5\n",
    "\n",
    "torch.manual_seed(40)\n",
    "x = torch.rand((500,20),dtype=torch.float32)\n",
    "y = torch.randint(low=0,high=2,size=(500,1),dtype=torch.float32)\n",
    "dataset = TensorDataset(x, y)\n",
    "dataloader = DataLoader(dataset, batch_size=20) \n",
    "\n",
    "net=Model0(20,1)\n",
    "#print(output)\n",
    "#criterion = nn.BCEWithLogitsLoss()\n",
    "criterion = nn.MSELoss()\n",
    "opt=optim.SGD(net.parameters() , lr=lr , momentum = gamma)\n",
    "\n",
    "for i in range(epochs):\n",
    "    for x1,y1 in dataloader:\n",
    "        output=net.forward(x1)\n",
    "        #y1=y1.view(x1.shape[0])\n",
    "        #.long()\n",
    "        loss = criterion(output,y1)#在PyTorch中,有些情况下目标向量(target)需要使用.long()方法转换为长整型tensor(long tensor),\n",
    "        opt.zero_grad()\n",
    "        loss.backward()\n",
    "        opt.step()\n",
    "        print(loss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 多分类的实现\n",
    "\n",
    "* 需要在模型中添加softmax层\n",
    "* 注意目标向量的形式"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch:0 , loss:1.1747212409973145\n",
      "epoch:0 , loss:1.151442289352417\n",
      "epoch:0 , loss:1.3114445209503174\n",
      "epoch:0 , loss:1.1314448118209839\n",
      "epoch:0 , loss:1.2314448356628418\n",
      "epoch:0 , loss:1.2314448356628418\n",
      "epoch:0 , loss:1.2114447355270386\n",
      "epoch:0 , loss:1.2514448165893555\n",
      "epoch:0 , loss:1.1914448738098145\n",
      "epoch:0 , loss:1.2514448165893555\n",
      "epoch:1 , loss:1.2514448165893555\n",
      "epoch:1 , loss:1.311444878578186\n",
      "epoch:1 , loss:1.1714447736740112\n",
      "epoch:1 , loss:1.2114447355270386\n",
      "epoch:1 , loss:1.2714447975158691\n",
      "epoch:1 , loss:1.1514447927474976\n",
      "epoch:1 , loss:1.1714447736740112\n",
      "epoch:1 , loss:1.1514447927474976\n",
      "epoch:1 , loss:1.2714447975158691\n",
      "epoch:1 , loss:1.1714447736740112\n",
      "epoch:2 , loss:1.2714447975158691\n",
      "epoch:2 , loss:1.1314448118209839\n",
      "epoch:2 , loss:1.191444754600525\n",
      "epoch:2 , loss:1.2714447975158691\n",
      "epoch:2 , loss:1.2914448976516724\n",
      "epoch:2 , loss:1.2314448356628418\n",
      "epoch:2 , loss:1.311444878578186\n",
      "epoch:2 , loss:1.1714448928833008\n",
      "epoch:2 , loss:1.2314448356628418\n",
      "epoch:2 , loss:1.0314449071884155\n",
      "epoch:3 , loss:1.2114447355270386\n",
      "epoch:3 , loss:1.3914448022842407\n",
      "epoch:3 , loss:1.091444730758667\n",
      "epoch:3 , loss:1.2514448165893555\n",
      "epoch:3 , loss:1.191444754600525\n",
      "epoch:3 , loss:1.2114447355270386\n",
      "epoch:3 , loss:1.1714448928833008\n",
      "epoch:3 , loss:1.1914448738098145\n",
      "epoch:3 , loss:1.2514448165893555\n",
      "epoch:3 , loss:1.1714448928833008\n",
      "epoch:4 , loss:1.2314448356628418\n",
      "epoch:4 , loss:1.2514448165893555\n",
      "epoch:4 , loss:1.2314448356628418\n",
      "epoch:4 , loss:1.2914446592330933\n",
      "epoch:4 , loss:1.2314448356628418\n",
      "epoch:4 , loss:1.1114448308944702\n",
      "epoch:4 , loss:1.191444754600525\n",
      "epoch:4 , loss:1.1314448118209839\n",
      "epoch:4 , loss:1.2714447975158691\n",
      "epoch:4 , loss:1.191444754600525\n"
     ]
    }
   ],
   "source": [
    "#确定数据、确定优先需要设置的值\n",
    "lr = 0.1\n",
    "gamma = 0.9\n",
    "epochs=5\n",
    "bs=50\n",
    "\n",
    "torch.manual_seed(420)\n",
    "X = torch.rand((500,20),dtype=torch.float32) * 100\n",
    "y = torch.randint(low=0,high=3,size=(500,1),dtype=torch.float32)\n",
    "\n",
    "data = TensorDataset(X,y)\n",
    "batchdata = DataLoader(data, batch_size=bs, shuffle = True)\n",
    "\n",
    "input_ = X.shape[1] #特征的数目\n",
    "output_ = len(y.unique()) #分类的数目\n",
    "\n",
    "torch.manual_seed(420)\n",
    "net = Model0(in_features=input_, out_features=output_)\n",
    "criterion = nn.CrossEntropyLoss()\n",
    "opt = optim.SGD(net.parameters() , lr=lr , momentum = gamma) #动量参数\n",
    "\n",
    "\n",
    "for i in range(epochs):\n",
    "    for x,y in batchdata:\n",
    "        y = y.view(x.shape[0]).long()\n",
    "        z1 = net.forward(x)\n",
    "        loss = criterion(z1,y)\n",
    "        opt.zero_grad()\n",
    "        loss.backward()\n",
    "        opt.step()\n",
    "        print(\"epoch:{} , loss:{}\".format(i,loss.item()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 基于天猫订单数据的简单推荐算法实现"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user</th>\n",
       "      <th>brand</th>\n",
       "      <th>behavr</th>\n",
       "      <th>date</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10944750</td>\n",
       "      <td>13451</td>\n",
       "      <td>0</td>\n",
       "      <td>06/04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10944750</td>\n",
       "      <td>13451</td>\n",
       "      <td>2</td>\n",
       "      <td>06/04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>10944750</td>\n",
       "      <td>13451</td>\n",
       "      <td>2</td>\n",
       "      <td>06/04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10944750</td>\n",
       "      <td>13451</td>\n",
       "      <td>0</td>\n",
       "      <td>06/04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10944750</td>\n",
       "      <td>13451</td>\n",
       "      <td>0</td>\n",
       "      <td>06/04</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       user  brand  behavr   date\n",
       "0  10944750  13451       0  06/04\n",
       "1  10944750  13451       2  06/04\n",
       "2  10944750  13451       2  06/04\n",
       "3  10944750  13451       0  06/04\n",
       "4  10944750  13451       0  06/04"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "r1=pd.read_csv(r\"D:\\t_alibaba_data3.txt\",names=[\"user\",\"brand\",\"behavr\",\"date\"],sep=\"\\t\",dtype={\"behavr\":int})\n",
    "r1.head()#行为数据要做机器学习是需要进行处理的"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>behavr</th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>user</th>\n",
       "      <th>brand</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">19500</th>\n",
       "      <th>143</th>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>949</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2257</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2610</th>\n",
       "      <td>20</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2641</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">12417500</th>\n",
       "      <th>24658</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24819</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28411</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29099</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29418</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>57655 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "behavr           0  1  2  3\n",
       "user     brand             \n",
       "19500    143     2  0  0  0\n",
       "         949     1  0  0  0\n",
       "         2257    1  0  0  0\n",
       "         2610   20  0  0  0\n",
       "         2641    3  0  0  0\n",
       "...             .. .. .. ..\n",
       "12417500 24658   1  0  0  0\n",
       "         24819   0  1  0  0\n",
       "         28411   0  1  0  0\n",
       "         29099   0  1  0  0\n",
       "         29418   1  0  0  0\n",
       "\n",
       "[57655 rows x 4 columns]"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "b1=r1.groupby([\"user\",\"brand\",\"behavr\"]).size().unstack(fill_value=0)\n",
    "b1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[2, 0, 0],\n",
       "       [1, 0, 0],\n",
       "       [1, 0, 0],\n",
       "       ...,\n",
       "       [0, 0, 0],\n",
       "       [0, 0, 0],\n",
       "       [1, 0, 0]], dtype=int64)"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z1=b1[[0,2,3]].values\n",
    "z1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "def com1(x):\n",
    "    if x>1:\n",
    "        return 1\n",
    "    else:\n",
    "        return x\n",
    "t1=b1[1].map(com1).values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, ..., 1, 1, 0], dtype=int64)"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[2, 0, 0],\n",
       "       [1, 0, 0],\n",
       "       [1, 0, 0],\n",
       "       ...,\n",
       "       [0, 0, 0],\n",
       "       [0, 0, 0],\n",
       "       [1, 0, 0]], dtype=int64)"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "features_z1=torch.tensor(z1,dtype=torch.float32)\n",
    "features_t1= torch.tensor(t1,dtype=torch.float32).unsqueeze(dim=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset=TensorDataset(features_z1, features_t1)\n",
    "dataloader = DataLoader(dataset, batch_size=10000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Mode_tb(nn.Module):\n",
    "    def __init__(self,in_features=3,out_features=1):\n",
    "        super(Mode_tb,self).__init__()\n",
    "        self.h1=nn.Linear(in_features,10,bias=True)\n",
    "        self.h2=nn.Linear(10,10,bias=True)\n",
    "        self.out=nn.Linear(10,out_features,bias=True)\n",
    "\n",
    "    def forward(self, x):\n",
    "        h1_out=self.h1(x)\n",
    "        h1_out_r=torch.relu(h1_out)\n",
    "        h2_out=self.h2(h1_out_r)\n",
    "        h2_out_r=torch.relu(h2_out)\n",
    "        out_out=self.out(h2_out_r)\n",
    "        #out_out_s=torch.softmax(out_out,dim=1)\n",
    "        out_out=torch.sigmoid(out_out)\n",
    "        return out_out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[-0.0510],\n",
       "        [-0.1054],\n",
       "        [-0.1054],\n",
       "        ...,\n",
       "        [-0.1054],\n",
       "        [-0.1054],\n",
       "        [ 0.0024]], grad_fn=<AddmmBackward0>)"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.],\n",
       "        [0.],\n",
       "        [0.],\n",
       "        ...,\n",
       "        [0.],\n",
       "        [0.],\n",
       "        [0.]])"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y1.unsqueeze(dim=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.6154, grad_fn=<BinaryCrossEntropyBackward0>) tensor(0.9204) tensor(0)\n",
      "tensor(0.6058, grad_fn=<BinaryCrossEntropyBackward0>) tensor(0.9157) tensor(0)\n",
      "tensor(0.5727, grad_fn=<BinaryCrossEntropyBackward0>) tensor(0.9223) tensor(0)\n",
      "tensor(0.5392, grad_fn=<BinaryCrossEntropyBackward0>) tensor(0.9280) tensor(0)\n",
      "tensor(0.5236, grad_fn=<BinaryCrossEntropyBackward0>) tensor(0.9209) tensor(0)\n",
      "tensor(0.4956, grad_fn=<BinaryCrossEntropyBackward0>) tensor(0.9219) tensor(0)\n",
      "tensor(0.4768, grad_fn=<BinaryCrossEntropyBackward0>) tensor(0.9204) tensor(0)\n",
      "tensor(0.4613, grad_fn=<BinaryCrossEntropyBackward0>) tensor(0.9157) tensor(0)\n",
      "tensor(0.4407, grad_fn=<BinaryCrossEntropyBackward0>) tensor(0.9230) tensor(9)\n",
      "tensor(0.4220, grad_fn=<BinaryCrossEntropyBackward0>) tensor(0.9288) tensor(11)\n",
      "tensor(0.4129, grad_fn=<BinaryCrossEntropyBackward0>) tensor(0.9234) tensor(26)\n",
      "tensor(0.4015, grad_fn=<BinaryCrossEntropyBackward0>) tensor(0.9236) tensor(18)\n",
      "tensor(0.3909, grad_fn=<BinaryCrossEntropyBackward0>) tensor(0.9214) tensor(20)\n",
      "tensor(0.3840, grad_fn=<BinaryCrossEntropyBackward0>) tensor(0.9185) tensor(36)\n",
      "tensor(0.3691, grad_fn=<BinaryCrossEntropyBackward0>) tensor(0.9238) tensor(25)\n",
      "tensor(0.3546, grad_fn=<BinaryCrossEntropyBackward0>) tensor(0.9300) tensor(27)\n",
      "tensor(0.3516, grad_fn=<BinaryCrossEntropyBackward0>) tensor(0.9244) tensor(38)\n",
      "tensor(0.3451, grad_fn=<BinaryCrossEntropyBackward0>) tensor(0.9238) tensor(22)\n",
      "tensor(0.3384, grad_fn=<BinaryCrossEntropyBackward0>) tensor(0.9218) tensor(25)\n",
      "tensor(0.3357, grad_fn=<BinaryCrossEntropyBackward0>) tensor(0.9190) tensor(46)\n",
      "tensor(0.3229, grad_fn=<BinaryCrossEntropyBackward0>) tensor(0.9244) tensor(31)\n",
      "tensor(0.3103, grad_fn=<BinaryCrossEntropyBackward0>) tensor(0.9305) tensor(35)\n",
      "tensor(0.3107, grad_fn=<BinaryCrossEntropyBackward0>) tensor(0.9248) tensor(46)\n",
      "tensor(0.3076, grad_fn=<BinaryCrossEntropyBackward0>) tensor(0.9236) tensor(23)\n",
      "tensor(0.3036, grad_fn=<BinaryCrossEntropyBackward0>) tensor(0.9224) tensor(31)\n",
      "tensor(0.3035, grad_fn=<BinaryCrossEntropyBackward0>) tensor(0.9198) tensor(56)\n",
      "tensor(0.2919, grad_fn=<BinaryCrossEntropyBackward0>) tensor(0.9246) tensor(40)\n",
      "tensor(0.2804, grad_fn=<BinaryCrossEntropyBackward0>) tensor(0.9309) tensor(44)\n",
      "tensor(0.2833, grad_fn=<BinaryCrossEntropyBackward0>) tensor(0.9253) tensor(55)\n",
      "tensor(0.2828, grad_fn=<BinaryCrossEntropyBackward0>) tensor(0.9242) tensor(31)\n"
     ]
    }
   ],
   "source": [
    "lr=0.1\n",
    "gamma=0.5\n",
    "epochs=5\n",
    "\n",
    "net=Mode_tb()\n",
    "#print(output)\n",
    "#criterion = nn.BCEWithLogitsLoss()\n",
    "criterion = nn.BCELoss()\n",
    "opt=optim.SGD(net.parameters() , lr=lr , momentum = gamma)\n",
    "\n",
    "for i in range(epochs):\n",
    "    for x1,y1 in dataloader:\n",
    "        output=net.forward(x1)\n",
    "        #y1=y1.view(x1.shape[0])\n",
    "        #.long()\n",
    "        \n",
    "        loss = criterion(output,y1)#在PyTorch中,有些情况下目标向量(target)需要使用.long()方法转换为长整型tensor(long tensor),\n",
    "        opt.zero_grad()\n",
    "        loss.backward()\n",
    "        opt.step()\n",
    "        #编程把output映射为0和1张量\n",
    "        output = torch.where(output>0.5,torch.tensor(1),torch.tensor(0))\n",
    "        #计算准确率\n",
    "        acc = torch.sum(output==y1)/y1.shape[0]\n",
    "        #计算f1-score\n",
    "        tp = torch.sum((output==1) & (y1==1))\n",
    "        print(loss,acc,tp)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 基于真实数据的神经网络模型构建\n",
    "\n",
    "‌FashionMNIST是一个包含10类服饰分类的数据集，共有7万个不同商品的正面图片。‌ 数据集的大小和格式与原始的MNIST完全一致，包含60000个训练图片和10000个测试图片，每张图片都是28x28的灰度图像‌，像素值在0到255之间，共分为10个类别，分别代表不同的服饰类型‌"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "mnist = torchvision.datasets.FashionMNIST(root='H:\\data\\FashionMNIST', train=True, download=True, transform=transforms.ToTensor())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Dataset FashionMNIST\n",
       "    Number of datapoints: 60000\n",
       "    Root location: H:\\data\\FashionMNIST\n",
       "    Split: Train\n",
       "    StandardTransform\n",
       "Transform: ToTensor()"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mnist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(mnist[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x1e949fe1b50>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaEAAAGdCAYAAAC7EMwUAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAhPklEQVR4nO3df3DU9b3v8dfm1xJwsxgh2URiTFuoChxaFflxkF9Xc0inXBV7LmpvD8xtHa3ADAcdW8o5I6dzhzh25HLnUumtp5fCVCpz5vrrFK4aDybIobSIeOWgw4klSCxJIxF2Q0g22eRz/+CSGkHM++uGT348HzM7Y3a/L78fvnyTV77s7ntDzjknAAA8yPC9AADA8EUJAQC8oYQAAN5QQgAAbyghAIA3lBAAwBtKCADgDSUEAPAmy/cCPq27u1snTpxQJBJRKBTyvRwAgJFzTi0tLSouLlZGxqWvdQZcCZ04cUIlJSW+lwEA+ILq6+s1bty4S24z4EooEolIkmbpG8pStufVAACsUurUHu3s+Xl+Kf1WQk899ZR+8pOfqKGhQRMnTtSGDRt06623fm7u/D/BZSlbWSFKCAAGnf8/kbQvT6n0ywsTtm/frpUrV2rNmjU6ePCgbr31VlVUVOj48eP9sTsAwCDVLyW0fv16ffe739X3vvc9XX/99dqwYYNKSkq0adOm/tgdAGCQSnsJdXR06MCBAyovL+91f3l5ufbu3XvB9slkUolEotcNADA8pL2ETp48qa6uLhUWFva6v7CwUI2NjRdsX1lZqWg02nPjlXEAMHz025tVP/2ElHPuok9SrV69WvF4vOdWX1/fX0sCAAwwaX913JgxY5SZmXnBVU9TU9MFV0eSFA6HFQ6H070MAMAgkPYroZycHN10002qqqrqdX9VVZVmzpyZ7t0BAAaxfnmf0KpVq/Sd73xHN998s2bMmKGf//znOn78uB588MH+2B0AYJDqlxJavHixmpub9eMf/1gNDQ2aNGmSdu7cqdLS0v7YHQBgkAo555zvRXxSIpFQNBrVXN3BxAQAGIRSrlPVelHxeFx5eXmX3JaPcgAAeEMJAQC8oYQAAN5QQgAAbyghAIA3lBAAwBtKCADgDSUEAPCGEgIAeEMJAQC8oYQAAN5QQgAAbyghAIA3lBAAwBtKCADgDSUEAPCGEgIAeEMJAQC8oYQAAN5QQgAAbyghAIA3lBAAwBtKCADgDSUEAPCGEgIAeEMJAQC8yfK9AGBACYXsGefSv46LyLwq35w59VcTAu0rb9u+QDmzAMc7lJVtzrjODnNmwAtyrgbVj+c4V0IAAG8oIQCAN5QQAMAbSggA4A0lBADwhhICAHhDCQEAvKGEAADeUEIAAG8oIQCAN5QQAMAbSggA4A0DTIFPCGVmmjMulTJnMr52gznz3gNX2PfTZo5IkrJbbzFnstq67ft59U1z5rIOIw0yYDXAOaSQ/Xrgch6HUJatKkLOSX38tuBKCADgDSUEAPCGEgIAeEMJAQC8oYQAAN5QQgAAbyghAIA3lBAAwBtKCADgDSUEAPCGEgIAeEMJAQC8YYAp8AnWQY1SsAGm9X812pz59ow3zJl//ehL5owkfRCOmTMu176frNtmmDMTnvqjOZM6dtyckSQ5Z48EOB+CyLzyymDBri57JJEwbe9c348BV0IAAG8oIQCAN2kvobVr1yoUCvW6xWL2S3sAwNDXL88JTZw4Ua+99lrP15lBPuQJADDk9UsJZWVlcfUDAPhc/fKcUG1trYqLi1VWVqZ77rlHR48e/cxtk8mkEolErxsAYHhIewlNmzZNW7du1SuvvKKnn35ajY2Nmjlzppqbmy+6fWVlpaLRaM+tpKQk3UsCAAxQaS+hiooK3X333Zo8ebJuu+027dixQ5K0ZcuWi26/evVqxePxnlt9fX26lwQAGKD6/c2qo0aN0uTJk1VbW3vRx8PhsMLhcH8vAwAwAPX7+4SSyaTee+89FRUV9feuAACDTNpL6JFHHlFNTY3q6ur0u9/9Tt/61reUSCS0ZMmSdO8KADDIpf2f4z788EPde++9OnnypMaOHavp06dr3759Ki0tTfeuAACDXNpL6Nlnn033/xK4bLrb2y/Lfjq+fsac+Vb0TXNmREanOSNJNRnd5swfd9lf2dr1F/bj8MH6iDnTfXCmOSNJV/2bfdhn3sEGc+bk7KvNmY9usg9XlaTCffbMla/9wbS96+6QTvZtW2bHAQC8oYQAAN5QQgAAbyghAIA3lBAAwBtKCADgDSUEAPCGEgIAeEMJAQC8oYQAAN5QQgAAbyghAIA3/f6hdoAXoVCwnLMPhTzzn6abM39zQ7U584fOsebMuJyPzRlJ+uviA/bQf7ZnNh6ZY860Ho2aMxmjgg37bJxu/z39j3fY/55cZ8qcufKtYD++M5b8yZxJdHzJtH2qs116sY/rMa8GAIA0oYQAAN5QQgAAbyghAIA3lBAAwBtKCADgDSUEAPCGEgIAeEMJAQC8oYQAAN5QQgAAbyghAIA3lBAAwBumaOPyCjrdegCb/oPfmzPzrni3H1ZyoasVbHp0q8sxZ053jTJnHrthhznz0YSIOdPpgv2o+8famebMmQBTvjNT9u+L6f/loDkjSXfn7zdnnvjfk03bp1xnn7flSggA4A0lBADwhhICAHhDCQEAvKGEAADeUEIAAG8oIQCAN5QQAMAbSggA4A0lBADwhhICAHhDCQEAvGGAKS4vF2yg5kBWe6bAnGnOu8KcaUyNNmeuyjxjzkhSJKPNnLk2+6Q581GXfRhpZna3OdPhMs0ZSfqHif9szrRfn23OZIe6zJmZI06YM5L01+/+jTkzSkcD7asvuBICAHhDCQEAvKGEAADeUEIAAG8oIQCAN5QQAMAbSggA4A0lBADwhhICAHhDCQEAvKGEAADeUEIAAG8YYAp8QWPD9iGhI0Kd5kxOKGXOnOi80pyRpNq2r5oz/56wD3JdUHjYnOkMMIw0U8EG5wYZLFqcfcqcaXf2oaf2M+icvyy0DyN9O+C++oIrIQCAN5QQAMAbcwnt3r1bCxcuVHFxsUKhkF544YVejzvntHbtWhUXFys3N1dz587V4cP2S24AwNBnLqHW1lZNmTJFGzduvOjjTzzxhNavX6+NGzdq//79isViuv3229XS0vKFFwsAGFrML0yoqKhQRUXFRR9zzmnDhg1as2aNFi1aJEnasmWLCgsLtW3bNj3wwANfbLUAgCElrc8J1dXVqbGxUeXl5T33hcNhzZkzR3v37r1oJplMKpFI9LoBAIaHtJZQY2OjJKmwsLDX/YWFhT2PfVplZaWi0WjPraSkJJ1LAgAMYP3y6rhQKNTra+fcBfedt3r1asXj8Z5bfX19fywJADAApfXNqrFYTNK5K6KioqKe+5uami64OjovHA4rHA6ncxkAgEEirVdCZWVlisViqqqq6rmvo6NDNTU1mjlzZjp3BQAYAsxXQmfOnNH777/f83VdXZ3efvtt5efn65prrtHKlSu1bt06jR8/XuPHj9e6des0cuRI3XfffWldOABg8DOX0Jtvvql58+b1fL1q1SpJ0pIlS/TLX/5Sjz76qNra2vTQQw/p1KlTmjZtml599VVFIpH0rRoAMCSEnHPBJvv1k0QioWg0qrm6Q1kh+1A/DHCf8QKVS0Yy7QMrXco+7FOSMq+0D/y857eH7PsJ2b/tPkrZf5EbnXnWnJGkmtP2AaaHm2PmzI+/+pI589bZa82Z4hz7UFEp2PE71jHGnBkfvvirhy/l/5yaYs5IUsmIj82ZV1fONm2fSrVrT/U/KB6PKy8v75LbMjsOAOANJQQA8IYSAgB4QwkBALyhhAAA3lBCAABvKCEAgDeUEADAG0oIAOANJQQA8IYSAgB4QwkBALyhhAAA3qT1k1WBzxVgaHsoy36aBp2iXf/d682Z+SP/2ZzZ2361OTM2q8Wc6XT2CeSSVBSOmzORwnZz5nTXSHMmP+uMOdPSlWvOSNLIjKQ5E+Tv6cack+bM3752ozkjSZFJzeZMXrbteqXbcH3DlRAAwBtKCADgDSUEAPCGEgIAeEMJAQC8oYQAAN5QQgAAbyghAIA3lBAAwBtKCADgDSUEAPCGEgIAeMMAU1xWoewcc6a73T4YM6gxhzrMmZNd2ebM6Iyz5kxOqMuc6Qg4wHRmfp0581GAIaFvtZWZM5HMNnNmbIZ9qKgklWTbh30eai8xZ3a2fsWc+e43XzNnJOnXP7/dnMl5ea9p+wzX2fdtrYsBACBdKCEAgDeUEADAG0oIAOANJQQA8IYSAgB4QwkBALyhhAAA3lBCAABvKCEAgDeUEADAG0oIAODN8B5gGgoFi2XZB1aGMgP0fYY9092etO+n2z4YMyjXaR8Qejn99/+50ZypT402Zxo77ZnRmfahp10Kdo7va4uaMyMy+j608ryxWQlzJtFtH5QaVEv3CHOmM8DQ2CDH7gdX1ZozkvRc/LZAuf7ClRAAwBtKCADgDSUEAPCGEgIAeEMJAQC8oYQAAN5QQgAAbyghAIA3lBAAwBtKCADgDSUEAPCGEgIAeDNkBpiGsux/FJdKBdpXkCGczj6fcEhqu+MWc6b+TvuA1W9//ffmjCQ1piLmzMGz15oz0cw2c2ZUhn04bbuzD9uVpBMdV5ozQYZw5medMWcKAgw97XLBft/+Y6f9OAQRZDjthyn7sZOklv/YYs6M3hpoV33ClRAAwBtKCADgjbmEdu/erYULF6q4uFihUEgvvPBCr8eXLl2qUCjU6zZ9+vR0rRcAMISYS6i1tVVTpkzRxo2f/eFfCxYsUENDQ89t586dX2iRAIChyfxsfkVFhSoqKi65TTgcViwWC7woAMDw0C/PCVVXV6ugoEATJkzQ/fffr6amps/cNplMKpFI9LoBAIaHtJdQRUWFnnnmGe3atUtPPvmk9u/fr/nz5yuZvPjLSysrKxWNRntuJSUl6V4SAGCASvv7hBYvXtzz35MmTdLNN9+s0tJS7dixQ4sWLbpg+9WrV2vVqlU9XycSCYoIAIaJfn+zalFRkUpLS1VbW3vRx8PhsMLhcH8vAwAwAPX7+4Sam5tVX1+voqKi/t4VAGCQMV8JnTlzRu+//37P13V1dXr77beVn5+v/Px8rV27VnfffbeKiop07Ngx/ehHP9KYMWN01113pXXhAIDBz1xCb775pubNm9fz9fnnc5YsWaJNmzbp0KFD2rp1q06fPq2ioiLNmzdP27dvVyRin8kFABjaQs4553sRn5RIJBSNRjVXdygrFGz44kCUVWR/31RnWaE58/H1I82Zs7GQOSNJX/vGe+bM0sI95sxHXXnmTHYo2HDalq5ccyaWfdqc2RW/wZy5Iss+wDTIoFRJujH3mDlzutt+7hVnnTJnfvD+t8yZwpH2oZ2S9I+l9jfad7puc+ZIp/158UiGfZCyJL1x9ivmzPM3jDVtn3KdqtaLisfjysu79Pcvs+MAAN5QQgAAbyghAIA3lBAAwBtKCADgDSUEAPCGEgIAeEMJAQC8oYQAAN5QQgAAbyghAIA3lBAAwBtKCADgTb9/surlkqyYas4UrDkaaF9fy/vQnLkh1z49ur3bPkV8REanOfNu29XmjCSd7c4xZ2o77NPE4yn7dObMkH2SsSQ1ddg/cuTJutvMmX+55WfmzN+dWGDOZOQGG5Lf3HWFOXP3FYkAe7Kf4w9cs9uc+VJOkzkjSb9ptX8Y54nOK82Zwuy4OXNt9kfmjCQtivy7OfO8bFO0LbgSAgB4QwkBALyhhAAA3lBCAABvKCEAgDeUEADAG0oIAOANJQQA8IYSAgB4QwkBALyhhAAA3lBCAABvBuwA01BWlkKhvi9v2rr95n38h8hhc0aSzrqwORNkGGmQQYhBRLPOBsolO+2nT1NnXqB9WU0INwbK3ZX3tjmze+M0c2ZW+wpz5g/zN5sz/9KWac5I0kcp+9/TPXXzzZm3jpeYM9OvrTNnJkf+aM5IwYbnRjLbzZnsUMqcae22/xySpH3t9uG0/YkrIQCAN5QQAMAbSggA4A0lBADwhhICAHhDCQEAvKGEAADeUEIAAG8oIQCAN5QQAMAbSggA4A0lBADwZsAOMG34/k3KDI/o8/Zro//DvI9tH083ZySpZMTH5kxpzklzZkruB+ZMEJEM+8BFSfpqnn3o4m9ax5kz1aevM2eKsk+bM5L0xtkvmzPPrv2JObP0bx82Z2bsfNCcSVwb7PfM1ChnzuRNaTZn/u7rO8yZnFCXOXO6yz6IVJLyw63mzOjMYAOBrYIMUpakSEabOZP51a+YtnddSam2b9tyJQQA8IYSAgB4QwkBALyhhAAA3lBCAABvKCEAgDeUEADAG0oIAOANJQQA8IYSAgB4QwkBALyhhAAA3gzYAaYjm7qVmdPd5+1/k/iaeR9fyv3InJGkk50Rc+aVM5PNmXG5p8yZaKZ9OOFXwo3mjCS93T7anHn5o4nmTHFuwpz5U2fUnJGk5s5R5szZbvsgyV/8t/XmzJN/us2cuSv/LXNGkqbk2IeRnu62/077bkfMnGnp7vtg4/PaXbY5I0nxAINPIwG+Bzud/Udxpuv7z8dPGp1hH7CamHyVaftUZzsDTAEAAx8lBADwxlRClZWVmjp1qiKRiAoKCnTnnXfqyJEjvbZxzmnt2rUqLi5Wbm6u5s6dq8OHD6d10QCAocFUQjU1NVq2bJn27dunqqoqpVIplZeXq7X1zx/89MQTT2j9+vXauHGj9u/fr1gspttvv10tLS1pXzwAYHAzPRv28ssv9/p68+bNKigo0IEDBzR79mw557RhwwatWbNGixYtkiRt2bJFhYWF2rZtmx544IH0rRwAMOh9oeeE4vG4JCk/P1+SVFdXp8bGRpWXl/dsEw6HNWfOHO3du/ei/49kMqlEItHrBgAYHgKXkHNOq1at0qxZszRp0iRJUmPjuZf6FhYW9tq2sLCw57FPq6ysVDQa7bmVlJQEXRIAYJAJXELLly/XO++8o1//+tcXPBYKhXp97Zy74L7zVq9erXg83nOrr68PuiQAwCAT6M2qK1as0EsvvaTdu3dr3LhxPffHYufeeNbY2KiioqKe+5uami64OjovHA4rHLa/2Q8AMPiZroScc1q+fLmee+457dq1S2VlZb0eLysrUywWU1VVVc99HR0dqqmp0cyZM9OzYgDAkGG6Elq2bJm2bdumF198UZFIpOd5nmg0qtzcXIVCIa1cuVLr1q3T+PHjNX78eK1bt04jR47Ufffd1y9/AADA4GUqoU2bNkmS5s6d2+v+zZs3a+nSpZKkRx99VG1tbXrooYd06tQpTZs2Ta+++qoiEfu8NQDA0BZyzjnfi/ikRCKhaDSq2bP+XllZfR9UOHXDAfO+/i1RbM5IUuEI+xtv/+KKD82ZI2ftwx1PtOWZMyOzOs0ZScrNtOdSzv5amIKw/XhfE7YP4JSkSIZ9+GROqMuc6QrwmqCJOSfMmeOpK80ZSWpMjTZn3j1r/366Mss+TPNQgO/bs6kcc0aSkl32p83bU/ZMNNxuzkzN/8CckaQM2X/kb3tpjmn77vZ2Hf2vaxSPx5WXd+mfScyOAwB4QwkBALyhhAAA3lBCAABvKCEAgDeUEADAG0oIAOANJQQA8IYSAgB4QwkBALyhhAAA3lBCAABvKCEAgDeBPln1csjY844yQtl93v6fXv1L8z7+/o5/Mmckqeb0debMbxonmzOJDvsnzo4d2WrO5GXbp1RLUn62fV/RAFOTR4RS5syp1ChzRpKSGX0/587r0sU/uv5SGpNRc+Zfu8ebM53dmeaMJCUD5IJMVf+4Y4w5U5wbN2daUn2fyP9Jx1ryzZmT8SvMmfaR9h/Fe7q+bM5I0oLYYXMmt8l2jncl+749V0IAAG8oIQCAN5QQAMAbSggA4A0lBADwhhICAHhDCQEAvKGEAADeUEIAAG8oIQCAN5QQAMAbSggA4E3IOed8L+KTEomEotGo5uoOZRkGmAYR//b0QLkvPXTEnLlldJ0581biGnPmeICBi53dwX4Xyc7oNmdGZneYMyMCDMbMyewyZyQpQ/Zvh+4AA0xHZdqPw6ispDmTl9VuzkhSJNOeywjZz4cgMgP8Hf0+fm36F/IZIgH+nlLO/j04I/oHc0aS/lfdTHMm+o33TdunXKeq9aLi8bjy8vIuuS1XQgAAbyghAIA3lBAAwBtKCADgDSUEAPCGEgIAeEMJAQC8oYQAAN5QQgAAbyghAIA3lBAAwBtKCADgzcAdYJqxyDbAtDvYwMrLpfXuaebMtB/tt2ci9qGG1+X8yZyRpGzZB1aOCDDkclSGfUBoe8DTOshvZXvaSsyZrgB72nXqenOmM8BgTEn609lLD528mOyAQ2Otup39fGhLBRuGHG8bYc5kZtjPvfbqMebMVe/aB/tKUnin/eeKFQNMAQCDAiUEAPCGEgIAeEMJAQC8oYQAAN5QQgAAbyghAIA3lBAAwBtKCADgDSUEAPCGEgIAeEMJAQC8GbgDTHWHbYApAgtNnRwo1xbLNWfCzUlzpqXUvp+8P7SaM5KUkUyZM93/971A+wKGKgaYAgAGBUoIAOCNqYQqKys1depURSIRFRQU6M4779SRI0d6bbN06VKFQqFet+nTp6d10QCAocFUQjU1NVq2bJn27dunqqoqpVIplZeXq7W197+/L1iwQA0NDT23nTt3pnXRAIChIcuy8csvv9zr682bN6ugoEAHDhzQ7Nmze+4Ph8OKxWLpWSEAYMj6Qs8JxeNxSVJ+fn6v+6urq1VQUKAJEybo/vvvV1NT02f+P5LJpBKJRK8bAGB4CFxCzjmtWrVKs2bN0qRJk3rur6io0DPPPKNdu3bpySef1P79+zV//nwlkxd/aW5lZaWi0WjPraSkJOiSAACDTOD3CS1btkw7duzQnj17NG7cuM/crqGhQaWlpXr22We1aNGiCx5PJpO9CiqRSKikpIT3CV1GvE/oz3ifEPDFWd4nZHpO6LwVK1bopZde0u7duy9ZQJJUVFSk0tJS1dbWXvTxcDiscDgcZBkAgEHOVELOOa1YsULPP/+8qqurVVZW9rmZ5uZm1dfXq6ioKPAiAQBDk+k5oWXLlulXv/qVtm3bpkgkosbGRjU2NqqtrU2SdObMGT3yyCP67W9/q2PHjqm6uloLFy7UmDFjdNddd/XLHwAAMHiZroQ2bdokSZo7d26v+zdv3qylS5cqMzNThw4d0tatW3X69GkVFRVp3rx52r59uyKRSNoWDQAYGsz/HHcpubm5euWVV77QggAAw0egFyZgaHH7DwXKjUjzOj5L3t7LtCNJ3ZdvVwDEAFMAgEeUEADAG0oIAOANJQQA8IYSAgB4QwkBALyhhAAA3lBCAABvKCEAgDeUEADAG0oIAOANJQQA8IYSAgB4QwkBALyhhAAA3lBCAABvKCEAgDeUEADAG0oIAOANJQQA8IYSAgB4QwkBALyhhAAA3lBCAABvsnwv4NOcc5KklDol53kxAACzlDol/fnn+aUMuBJqaWmRJO3RTs8rAQB8ES0tLYpGo5fcJuT6UlWXUXd3t06cOKFIJKJQKNTrsUQioZKSEtXX1ysvL8/TCv3jOJzDcTiH43AOx+GcgXAcnHNqaWlRcXGxMjIu/azPgLsSysjI0Lhx4y65TV5e3rA+yc7jOJzDcTiH43AOx+Ec38fh866AzuOFCQAAbyghAIA3g6qEwuGwHnvsMYXDYd9L8YrjcA7H4RyOwzkch3MG23EYcC9MAAAMH4PqSggAMLRQQgAAbyghAIA3lBAAwJtBVUJPPfWUysrKNGLECN1000164403fC/pslq7dq1CoVCvWywW872sfrd7924tXLhQxcXFCoVCeuGFF3o97pzT2rVrVVxcrNzcXM2dO1eHDx/2s9h+9HnHYenSpRecH9OnT/ez2H5SWVmpqVOnKhKJqKCgQHfeeaeOHDnSa5vhcD705TgMlvNh0JTQ9u3btXLlSq1Zs0YHDx7UrbfeqoqKCh0/ftz30i6riRMnqqGhoed26NAh30vqd62trZoyZYo2btx40cefeOIJrV+/Xhs3btT+/fsVi8V0++2398whHCo+7zhI0oIFC3qdHzt3Dq0ZjDU1NVq2bJn27dunqqoqpVIplZeXq7W1tWeb4XA+9OU4SIPkfHCDxC233OIefPDBXvddd9117oc//KGnFV1+jz32mJsyZYrvZXglyT3//PM9X3d3d7tYLOYef/zxnvva29tdNBp1P/vZzzys8PL49HFwzrklS5a4O+64w8t6fGlqanKSXE1NjXNu+J4Pnz4Ozg2e82FQXAl1dHTowIEDKi8v73V/eXm59u7d62lVftTW1qq4uFhlZWW65557dPToUd9L8qqurk6NjY29zo1wOKw5c+YMu3NDkqqrq1VQUKAJEybo/vvvV1NTk+8l9at4PC5Jys/PlzR8z4dPH4fzBsP5MChK6OTJk+rq6lJhYWGv+wsLC9XY2OhpVZfftGnTtHXrVr3yyit6+umn1djYqJkzZ6q5udn30rw5//c/3M8NSaqoqNAzzzyjXbt26cknn9T+/fs1f/58JZNJ30vrF845rVq1SrNmzdKkSZMkDc/z4WLHQRo858OAm6J9KZ/+aAfn3AX3DWUVFRU9/z158mTNmDFDX/7yl7VlyxatWrXK48r8G+7nhiQtXry4578nTZqkm2++WaWlpdqxY4cWLVrkcWX9Y/ny5XrnnXe0Z8+eCx4bTufDZx2HwXI+DIoroTFjxigzM/OC32Sampou+I1nOBk1apQmT56s2tpa30vx5vyrAzk3LlRUVKTS0tIheX6sWLFCL730kl5//fVeH/0y3M6HzzoOFzNQz4dBUUI5OTm66aabVFVV1ev+qqoqzZw509Oq/Esmk3rvvfdUVFTkeynelJWVKRaL9To3Ojo6VFNTM6zPDUlqbm5WfX39kDo/nHNavny5nnvuOe3atUtlZWW9Hh8u58PnHYeLGbDng8cXRZg8++yzLjs72/3iF79w7777rlu5cqUbNWqUO3bsmO+lXTYPP/ywq66udkePHnX79u1z3/zmN10kEhnyx6ClpcUdPHjQHTx40Ely69evdwcPHnQffPCBc865xx9/3EWjUffcc8+5Q4cOuXvvvdcVFRW5RCLheeXpdanj0NLS4h5++GG3d+9eV1dX515//XU3Y8YMd/XVVw+p4/D973/fRaNRV11d7RoaGnpuZ8+e7dlmOJwPn3ccBtP5MGhKyDnnfvrTn7rS0lKXk5Pjbrzxxl4vRxwOFi9e7IqKilx2drYrLi52ixYtcocPH/a9rH73+uuvO0kX3JYsWeKcO/ey3Mcee8zFYjEXDofd7Nmz3aFDh/wuuh9c6jicPXvWlZeXu7Fjx7rs7Gx3zTXXuCVLlrjjx4/7XnZaXezPL8lt3ry5Z5vhcD583nEYTOcDH+UAAPBmUDwnBAAYmighAIA3lBAAwBtKCADgDSUEAPCGEgIAeEMJAQC8oYQAAN5QQgAAbyghAIA3lBAAwBtKCADgzf8DCTTz4LFHB6oAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(mnist[0][0].view((28, 28)).numpy())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([60000, 28, 28])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mnist.targets\n",
    "mnist.data.shape\n",
    "#mnist[0][0].view((28, 28))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "#定义神经网路的架构\n",
    "class Model_mnist(nn.Module):\n",
    "    def __init__(self,in_features=10,out_features=2):\n",
    "        super().__init__() \n",
    "        #self.normalize = nn.BatchNorm2d(num_features=1)\n",
    "        self.linear1 = nn.Linear(in_features,128,bias=False)\n",
    "        self.output = nn.Linear(128,out_features,bias=False)\n",
    "\n",
    "    def forward(self, x):\n",
    "        #x = self.normalize(x)\n",
    "        x = x.view(-1, 28*28)\n",
    "        #需要对数据的结构进行一个改变，这里的“-1”代表，我不想算，请pytorch帮我计算\n",
    "        sigma1 = torch.relu(self.linear1(x))\n",
    "        z2 = self.output(sigma1)\n",
    "        return z2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "lr = 0.15\n",
    "gamma = 0.1\n",
    "epochs = 5\n",
    "bs = 128\n",
    "batchdata = DataLoader(mnist,batch_size=bs, shuffle = True)\n",
    "input_ = mnist.data[0].numel()\n",
    "output_ = len(mnist.targets.unique())\n",
    "\n",
    "net = Model_mnist(in_features=input_, out_features=output_)\n",
    "criterion = nn.CrossEntropyLoss() #定义损失函数\n",
    "opt = optim.SGD(net.parameters(), lr=lr,momentum=gamma) #定义优化算法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(2.3283, grad_fn=<NllLossBackward0>)\n",
      "tensor(0.7433, grad_fn=<NllLossBackward0>)\n",
      "tensor(0.4928, grad_fn=<NllLossBackward0>)\n",
      "tensor(0.5021, grad_fn=<NllLossBackward0>)\n",
      "tensor(0.4844, grad_fn=<NllLossBackward0>)\n",
      "tensor(0.4970, grad_fn=<NllLossBackward0>)\n",
      "tensor(0.4970, grad_fn=<NllLossBackward0>)\n",
      "tensor(0.4291, grad_fn=<NllLossBackward0>)\n",
      "tensor(0.5489, grad_fn=<NllLossBackward0>)\n",
      "tensor(0.4384, grad_fn=<NllLossBackward0>)\n",
      "tensor(0.3684, grad_fn=<NllLossBackward0>)\n",
      "tensor(0.5305, grad_fn=<NllLossBackward0>)\n",
      "tensor(0.4032, grad_fn=<NllLossBackward0>)\n",
      "tensor(0.4254, grad_fn=<NllLossBackward0>)\n",
      "tensor(0.4315, grad_fn=<NllLossBackward0>)\n",
      "tensor(0.4283, grad_fn=<NllLossBackward0>)\n",
      "tensor(0.3779, grad_fn=<NllLossBackward0>)\n",
      "tensor(0.4869, grad_fn=<NllLossBackward0>)\n",
      "tensor(0.3957, grad_fn=<NllLossBackward0>)\n",
      "tensor(0.3591, grad_fn=<NllLossBackward0>)\n",
      "tensor(0.3423, grad_fn=<NllLossBackward0>)\n",
      "tensor(0.3246, grad_fn=<NllLossBackward0>)\n",
      "tensor(0.2931, grad_fn=<NllLossBackward0>)\n",
      "tensor(0.3662, grad_fn=<NllLossBackward0>)\n",
      "tensor(0.3042, grad_fn=<NllLossBackward0>)\n"
     ]
    }
   ],
   "source": [
    "for epoch in range(epochs):\n",
    "    for batch_idx, (x,y) in enumerate(batchdata):\n",
    "        y = y.view(x.shape[0])\n",
    "        sigma = net.forward(x)\n",
    "        loss = criterion(sigma,y)\n",
    "        opt.zero_grad()\n",
    "        loss.backward()\n",
    "        opt.step()\n",
    "        if batch_idx % 100 == 0: print(loss)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.19"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
